aitools list: emit JSON via --output json#5233
Conversation
b3b6d1a to
9df5e93
Compare
c02b45c to
c4ef02c
Compare
|
👋 @simonfaltum — Claude here on James's behalf. Rebased onto the new tip of #5234 (which itself was rebased onto current Drift fixes applied during the rebase:
Stacking note: PR base is set to Ready for review whenever you've got time. (comment posted by Claude) |
7666312 to
24d5351
Compare
c4ef02c to
e71f268
Compare
## Summary Promotes the **aitools skills-management surface** out of `experimental/` so the stable half lives at `databricks aitools …` and slots in next to the other top-level command groups. The matching **interface changes** (`--scope` enum, `--project`/`--global` deprecation, `--agents` auto-detect doc) live in a stacked follow-up: **#5234**. This is mostly a move, but it is not move-only — see [Non-move changes](#non-move-changes) below. - Source files for `install`, `update`, `uninstall`, `list`, `version` (and the agents/installer libs they depend on) physically move from `experimental/aitools/` to `cmd/aitools/` + `libs/aitools/`, matching the existing `cmd/apps/` + `libs/apps/` layout. OWNERS, Taskfile, and the pr-checklist skill are updated to match. - The top-level command is registered at `databricks aitools …`. - **Keeps the `tools` subtree under `experimental/aitools/`** — `query`, `discover-schema`, `get-default-warehouse`, `statement …` — because `tools.go` still says "There are no stability guarantees for these tools". - The old paths under `databricks experimental aitools install/update/uninstall/list/version` and `databricks experimental aitools skills install/list` keep working as **deprecated backward-compat aliases** that print a notice pointing at the new path (via cobra's `Deprecated` field). The aitools skills-management surface is feature-complete after the 5-PR series (#4810–#4814) that added state tracking, lifecycle commands, and project scope support. The `tools` subtree is functionally useful but its shape is still in flux, so promoting only the stable half. ## Non-move changes In addition to the file moves, this PR: - Removes the redundant `aitools/README.md` (apps/pipelines don't carry one); the same info lives in the command's Long description. - Rewrites the Long description on the new top-level `databricks aitools` command. - Adds a deprecation notice to every legacy alias under `databricks experimental aitools` (Lennart's review ask — they used to forward silently). - Refactors how the legacy `experimental aitools skills install [name]` wrapper is wired: the wrapper now lives in `cmd/aitools/legacy_skills.go` alongside the install code it wraps, and the previously-exported test-injection vars (`InstallSkillsForAgentsFn`, `ListSkillsFn`, `PromptAgentSelection`) are back to unexported. The two now-empty experimental files (`experimental/aitools/cmd/skills.go` and `skills_test.go`) are deleted. ## What's not in this PR These are deliberately separated and reviewed independently: - **#5234** — `--scope=project|global|both` flag, deprecation of `--project`/`--global` via `cobra.Deprecated`, `--agents` auto-detect help text. - **#5233** (draft) — `--output json` on `databricks aitools list`. ## Command shape after this PR ``` # Stable, top-level databricks aitools install # use --skills <name>[,<name>...] for specific skills databricks aitools update databricks aitools uninstall databricks aitools list databricks aitools version # Backward-compat aliases (print deprecation notice; point at the new paths) databricks experimental aitools install/update/uninstall/list/version databricks experimental aitools skills {list,install} # Experimental, unchanged path databricks experimental aitools tools query databricks experimental aitools tools discover-schema databricks experimental aitools tools get-default-warehouse databricks experimental aitools tools statement {submit,get,status,cancel} ``` ## Test plan - [x] `databricks aitools --help` shows install/update/uninstall/list/version (no `tools`) - [x] `databricks --help` lists `aitools` in the output - [x] `databricks experimental aitools install` prints a deprecation notice and still forwards - [x] `databricks experimental aitools tools query …` runs as before - [x] `databricks experimental aitools tools --help` lists query/discover-schema/get-default-warehouse/statement - [x] Existing aitools tests pass; the legacy-wrapper tests moved with the wrapper to `cmd/aitools/legacy_skills_test.go` This pull request was AI-assisted by Isaac. --------- Co-authored-by: simon <simon.faltum@databricks.com> Co-authored-by: simon <4305831+simonfaltum@users.noreply.github.com>
simonfaltum
left a comment
There was a problem hiding this comment.
Looks good after the fixes on this branch.
I also merged the latest stacked base/main, resolved the changelog/list conflicts, restored manifest fallback behavior, and added coverage for preserving scoped text labels while adding JSON output.
Teaches list to render as a structured {release, skills[...], summary{}}
document when --output json is passed. Text rendering is unchanged.
Stacked-on-#5234 rebased onto main now that #5234 has merged. The branch
state was carrying stale rehashes of the scope-flag work; squashed onto
current main to keep only the JSON-output delta.
Co-authored-by: Isaac
b643296 to
f8f0e15
Compare
…resolve # Conflicts: # cmd/aitools/list.go
) ## Summary Promotes the **aitools skills-management surface** out of `experimental/` so the stable half lives at `databricks aitools …` and slots in next to the other top-level command groups. The matching **interface changes** (`--scope` enum, `--project`/`--global` deprecation, `--agents` auto-detect doc) live in a stacked follow-up: **databricks#5234**. This is mostly a move, but it is not move-only — see [Non-move changes](#non-move-changes) below. - Source files for `install`, `update`, `uninstall`, `list`, `version` (and the agents/installer libs they depend on) physically move from `experimental/aitools/` to `cmd/aitools/` + `libs/aitools/`, matching the existing `cmd/apps/` + `libs/apps/` layout. OWNERS, Taskfile, and the pr-checklist skill are updated to match. - The top-level command is registered at `databricks aitools …`. - **Keeps the `tools` subtree under `experimental/aitools/`** — `query`, `discover-schema`, `get-default-warehouse`, `statement …` — because `tools.go` still says "There are no stability guarantees for these tools". - The old paths under `databricks experimental aitools install/update/uninstall/list/version` and `databricks experimental aitools skills install/list` keep working as **deprecated backward-compat aliases** that print a notice pointing at the new path (via cobra's `Deprecated` field). The aitools skills-management surface is feature-complete after the 5-PR series (databricks#4810–databricks#4814) that added state tracking, lifecycle commands, and project scope support. The `tools` subtree is functionally useful but its shape is still in flux, so promoting only the stable half. ## Non-move changes In addition to the file moves, this PR: - Removes the redundant `aitools/README.md` (apps/pipelines don't carry one); the same info lives in the command's Long description. - Rewrites the Long description on the new top-level `databricks aitools` command. - Adds a deprecation notice to every legacy alias under `databricks experimental aitools` (Lennart's review ask — they used to forward silently). - Refactors how the legacy `experimental aitools skills install [name]` wrapper is wired: the wrapper now lives in `cmd/aitools/legacy_skills.go` alongside the install code it wraps, and the previously-exported test-injection vars (`InstallSkillsForAgentsFn`, `ListSkillsFn`, `PromptAgentSelection`) are back to unexported. The two now-empty experimental files (`experimental/aitools/cmd/skills.go` and `skills_test.go`) are deleted. ## What's not in this PR These are deliberately separated and reviewed independently: - **databricks#5234** — `--scope=project|global|both` flag, deprecation of `--project`/`--global` via `cobra.Deprecated`, `--agents` auto-detect help text. - **databricks#5233** (draft) — `--output json` on `databricks aitools list`. ## Command shape after this PR ``` # Stable, top-level databricks aitools install # use --skills <name>[,<name>...] for specific skills databricks aitools update databricks aitools uninstall databricks aitools list databricks aitools version # Backward-compat aliases (print deprecation notice; point at the new paths) databricks experimental aitools install/update/uninstall/list/version databricks experimental aitools skills {list,install} # Experimental, unchanged path databricks experimental aitools tools query databricks experimental aitools tools discover-schema databricks experimental aitools tools get-default-warehouse databricks experimental aitools tools statement {submit,get,status,cancel} ``` ## Test plan - [x] `databricks aitools --help` shows install/update/uninstall/list/version (no `tools`) - [x] `databricks --help` lists `aitools` in the output - [x] `databricks experimental aitools install` prints a deprecation notice and still forwards - [x] `databricks experimental aitools tools query …` runs as before - [x] `databricks experimental aitools tools --help` lists query/discover-schema/get-default-warehouse/statement - [x] Existing aitools tests pass; the legacy-wrapper tests moved with the wrapper to `cmd/aitools/legacy_skills_test.go` This pull request was AI-assisted by Isaac. --------- Co-authored-by: simon <simon.faltum@databricks.com> Co-authored-by: simon <4305831+simonfaltum@users.noreply.github.com>
## Release v1.0.0 ### Notable Changes * The Databricks CLI is now generally available with version v1.0.0 as the first major release 🚀. From this version on, the CLI follows semantic versioning (see [README](README.md)). This change does not impact DABs or other existing commands beyond the changes listed below. * The 0.299.x line continues to receive security-critical patches through May 20, 2027; see [SECURITY](SECURITY.md) for the support policy. * Starting with v1.0.0, the CLI will use [immutable release tags](https://docs.github.com/en/code-security/concepts/supply-chain-security/immutable-releases) to increase security against supply chain attacks. * Breaking change: OAuth tokens for interactive logins (`auth_type = databricks-cli`) are now stored in the OS-native secure store by default (Keychain on macOS, Credential Manager on Windows, Secret Service on Linux) instead of `~/.databricks/token-cache.json`. After upgrading, run `databricks auth login` once per profile to re-authenticate; cached tokens from older versions are not migrated. To keep the previous file-backed storage, set `DATABRICKS_AUTH_STORAGE=plaintext` or add `auth_storage = plaintext` under `[__settings__]` in `~/.databrickscfg` (the env var takes precedence over the config setting), then re-run `databricks auth login`. On systems where the OS keyring is not reachable (e.g. Linux containers without a D-Bus session bus), the CLI transparently falls back to the file cache when reading tokens so legacy `token-cache.json` entries remain accessible without manual configuration. ### CLI * Added `databricks aitools` command group for installing Databricks skills into your coding agents (Claude Code, Cursor, Codex CLI, OpenCode, GitHub Copilot, Antigravity). Skills are fetched from [github.com/databricks/databricks-agent-skills](https://github.com/databricks/databricks-agent-skills) and either symlinked into each agent's skills directory or copied into the current project. Use `databricks aitools install` to set up, `update` to pull newer versions, `list` to see what's available, and `uninstall` to remove them. Pick where they go with `--scope=project|global` (`--scope=both` is accepted on `update` and `list`). * `[__settings__].default_profile` is now consulted as a fallback by `databricks api`, `databricks auth token`, and bundle commands when neither `--profile` nor `DATABRICKS_CONFIG_PROFILE` is set. `databricks auth token` continues to give precedence to `DATABRICKS_HOST` over `default_profile`. For bundle commands, `default_profile` only applies when the bundle does not pin its own `workspace.host`. * Fixed bug where auth commands did not load the DEFAULT profile properly during auth where type is `databricks-cli`. * `databricks workspace import-dir` now skips `.git`, `.databricks`, and `node_modules` directories during recursive imports. To import one of these directories deliberately, pass it as `SOURCE_PATH` ([#5118](#5118)). * `databricks postgres create-role --help` now documents the `--json` body shape and rejects the common mistake of wrapping the body in `{"role": ...}` client-side with a hint pointing at the correct shape ([#5111](#5111)). * `databricks aitools list` honors `--output json`, emitting a structured `{release, skills[...], summary{}}` document so coding agents and CI can consume the skill/version/installation matrix without scraping the tabular text output ([#5233](#5233)). ### Bundles * Make sure warnings asking for approval are understood by agents ([#5239](#5239)) * Support `replace_existing: true` on `postgres_branches` and `postgres_endpoints` so bundles can manage the implicitly-created production branch and primary read-write endpoint of a Lakebase project. * Add `postgres_catalogs` resource to bind a Unity Catalog catalog to a Postgres database on a Lakebase Autoscaling branch ([#5265](#5265)). * Add `postgres_synced_tables` resource to sync a Unity Catalog Delta table into a Postgres table on a Lakebase Autoscaling branch ([#5268](#5268)). * engine/direct: Changes to state file now persisted to .wal file right away instead of being saved in the end ([#5149](#5149))
Summary
databricks aitools listlearns--output json, emitting a structured document so coding agents and CI can consume the skill/version/installation matrix without scraping the tabwriter text output. Text rendering is unchanged.Stacked on #4917 (uses
--scopeand the moved-to-top-levelaitools/package). Base will rebase tomainonce #4917 merges.JSON shape
{ "release": "0.1.0", "skills": [ { "name": "databricks-jobs", "latest_version": "1.0.0", "experimental": false, "installed": { "global": "1.0.0", "project": "0.9.0" } } ], "summary": { "global": { "installed": 5, "total": 10 }, "project": { "installed": 3, "total": 10 } } }installedis keyed by scope; absent key = not installed in that scope; empty map = not installed anywhere.summaryonly includes scopes that were queried, so--scope=globalnarrows it to one key.releaseis the version string without thevprefix.This is the documented public contract — field names and types should not change without a major version bump.
Why
aitools listis one of the surfaces an agent reaches for first ("what's installed, what's available, what's stale"). Scraping tabwriter columns from stderr is fragile; a stable JSON contract makes the command declarative for non-human callers. Matches the convention used by other CLI commands that already honor--output json(bundle validate,pipelines run, etc.).Test plan
databricks aitools list --output jsonagainst a workspace with a mix of installed/uninstalled skills, both scopes — JSON validates against the shape above.databricks aitools list --output json --scope=global—summaryonly containsglobal.databricks aitools list(no--output) — output is byte-for-byte unchanged from main.TestRenderListJSON,TestRenderListJSONScopeFiltersSummary,TestInstalledStatusFromEntrycover the rendering paths.This pull request was AI-assisted by Isaac.